NASTransfer: Analyzing Architecture Transferability in Large Scale Neural Architecture Search
نویسندگان
چکیده
Neural Architecture Search (NAS) is an open and challenging problem in machine learning. While NAS offers great promise, the prohibitive computational demand of most existing methods makes it difficult to directly search architectures on large-scale tasks. The typical way conducting large scale for architectural building block a small dataset (either using proxy set from or completely different dataset) then transfer larger dataset. Despite number recent results that show promise datasets, comprehensive evaluation studying impact source datasets has not yet been addressed. In this work, we propose analyze architecture transferability by performing series experiments benchmarks such as ImageNet1K ImageNet22K. We find that: (i) size domain does seem influence performance target On average, searched (e.g., CIFAR10) perform similarly datasets. However, design sets considerable rankings methods. (ii) similar CIFAR10), they significantly differ ImageNet1K). (iii) Even random sampling baseline very competitive, but choice appropriate combination strategy can provide significant improvement over it. believe our extensive empirical analysis will prove useful future algorithms.
منابع مشابه
Progressive Neural Architecture Search
We propose a method for learning CNN structures that is more efficient than previous approaches: instead of using reinforcement learning (RL) or genetic algorithms (GA), we use a sequential model-based optimization (SMBO) strategy, in which we search for architectures in order of increasing complexity, while simultaneously learning a surrogate function to guide the search, similar to A* search....
متن کاملLarge-Scale Structures in Ilkhanid Architecture and Gothic Style of Europe
The mutual experience of European architecture during the transmission phase of Romanesque (around 11th and 12th centuries) to Gothic style simultaneous with Mongol invasions and political revolution from Seljuk empire to Ilkhanids have been assumed as a turning point through Iranian and European architecture. Thus, clarifying its mutual properties is one of the controversial discussion topics ...
متن کاملIs P2P a Suitable Architecture for Large-Scale Web Search?
In this note we discuss the feasibility of a peer-to-peer based infrastructure for large-scale web search that could provide an alternative to current centralized engines such as Google. We first outline the structure and performance bottlenecks of current search engines, and then discuss how to construct a peer-to-peer based engine. Our contention is that for the time being, a peer-to-peer sol...
متن کاملSystems and Architecture Support for Large-Scale Video Search
To index and search all of the videos in the Internet requires massive resources. Using commodity parts and existing libraries, a conservative estimate for a system requires 32 petabytes to store feature data, 480,000 CPU cores to build and update the index, and 1000 servers to service 1 million concurrent clients at a latency on the order of seconds. Such requirements in both power consumption...
متن کاملDifferentiable Neural Network Architecture Search
The successes of deep learning in recent years has been fueled by the development of innovative new neural network architectures. However, the design of a neural network architecture remains a difficult problem, requiring significant human expertise as well as computational resources. In this paper, we propose a method for transforming a discrete neural network architecture space into a continu...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2021
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v35i10.17121